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ABSTRACT 


As  an  approach  to  solving  nonlinear  programs,  we  study  a  class 
of  functions  known  to  be  exact  penalty  functions  for  a  proper  choice  of 
the  parameters.  The  goal  is  to  iteratively  determine  the  correct  param- 
eter values.  A  basic  algorithm  has  been  developed.  We  have  proved  that 
this  algorithm  converges  for  concave  programs,  and  in  the  limited  compu- 
tational tests  performed  to  date  it  has  always  converged  for  nonconcave 
programs  also.   Suggestions  for  continuing  the  work  are  given. 
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I .   Introduction. 

In  recent  years  the  technique  of  solving  nonlinear  programming 
problems  using  penalty  functions  has  been  extensively  developed.   The 
usual  strategy  is  to  convert  a  constrained  nonlinear  program  to  a 
sequence  of  unconstrained  optimizations  of  "penalty  functions."  These 
penalty  functions  are  constructed  so  that  the  sequence  of  unconstrained 
optimal  solutions  approaches  the  constrained  optimal  solution  for  the 
original  nonlinear  program.   (See,  for  example,  Fiacco  and  McCormick's 
book  [3].) 

This  strategy  may  involve  severe  computational  difficulties. 
Murray  [6]  has  shown  for  several  penalty  function  methods  that  the 
successive  unconstrained  problems  become  increasingly  ill  conditioned 
(and  hence  difficult  to  optimize)  since  a  parameter  must  approach 
infinity  to  force  the  unconstrained  optima  to  approach  the  solution 
of  the  nonlinear  program. 

An  alternate  strategy  is  to  try  to  find  a  single  function, 
called  an  "exact  penalty  function"  such  that  the  unconstrained  optimal 
solution  to  the  exact  penalty  function  is  exactly  the  optimal  solution 
to  the  nonlinear  program.   Then  a  single  unconstrained  optimization 
of  the  exact  penalty  function  will  solve  the  nonlinear  program.   For 
the  equality  constrained  problem,  Fletcher  [4]  has  found  a  continuously 
differentiable  exact  penalty  function  with  a  single  parameter  for 
which  a  correct  parameter  choice  is  not  difficult.   Similar  results 
have  been  developed  for  the  inequality  constrained  nonlinear  program 


by  Zangwill  [9],  Pietrzykowski  [8],  and  Evans  et  al  [2].   In  each 
of  these  cases,  however,  the  exact  penalty  functions  studied  are  not 
continuously  differentiable.   In  fact,  derivatives  fail  to  exist  along 
all  constraint  boundaries,  and  this  is  critical  since  in  most  nonlinear 
programs  the  optimal  solution  will  occur  at  the  boundary  of  the 
feasible  region.   The  problem  of  finding  a  continuously  differentiable 
exact  penalty  function  for  inequality  constrained  nonlinear  programs 
is  still  not  completely  solved. 

In  this  paper  we  examine  a  class  of  functions  which  are  known 
to  be  continuously  differentiable  exact  penalty  functions  _if  the  param- 
eters are  chosen  correctly.   We  then  consider  iterative  procedures 
for  selecting  the  proper  parameter  values.  The  result  is  a  solution 
strategy  which  involves  a  sequence  of  unconstrained  optimizations 
whose  solutions  converge  to  the  optimal  solution  of  the  original  non- 
linear program.  Thus  the  goal  of  only  having  to  perform  a  single 
unconstrained  optimization  is  not  achieved.   However,  the  successive 
unconstrained  optimizations  do  not  become  increasingly  ill-conditioned 
since  moderate  values  of  all  parameters  involved  suffice  to  give  a 
solution  to  the  original  nonlinear  program.   Hence,  this  approach 
may  offer  an  improvement  over  conventional  penalty  function  schemes. 


II.   The  Method. 

Throughout  this  discussion  we  will  consider  the  following 
inequality  constrained  nonlinear  program: 

(NLP)     maximize     f(x) 

subject  to  g . (x)  £  0     i  =  l,...,m  (1) 

x  e  E 

with  Lagrangian  function 


m 


L(x,A)  =  f(x)  -  I     A  g  (x). 
1=1 


Suppose  f   and  g.   (i  =  l,...,m)   are  twice  continuously  differen- 

tiable.   For  simplicity  it  will  be  assumed  that  NLP  has  at  least  a 

local  solution  x* ,   and  that  a  constraint  qualification  applies  so 

that  the  Kuhn-Tucker  necessary  conditions  hold  at  x*  with  optimal 

multiplier  vector   A*  e  E  .   Let   S   denote  the  feasible  region  for 

NLP(l):   S  =  {x  e  En  |  g.(x)  ^  0,  i  =  l,...,m}.   Let  E™  =  {A  e   Em  |  A .  k  0, 

i  =  1, . . . ,m}. 

a)   The  exact  penalty  function. 

Consider  the  function  of  x  e  E  ,   k  e   E  ,   A  c  E 

m 
P(x,k,A)  =  f(x)  -  (1/k)  I     A  [exp(k  g  (x))  -  1]      (2) 

i=l 


Note  that  for  X.  >  0,   k  >  0   the  exponential  term  imposes  a  penalty 
for  values  of  x  which  violate  the  constraint  g . (x)  i  0. 

The  following  theorem  has  been  proved  by  Gould  and  Howe  [5]: 

Theorem  1.   Suppose  x*  e  E   is  a  local  solution  for  the  nonlinear 
program  (1)  and  X*  e  E  ,   are  multipliers  such  that 

Al.    V  L(x*,X*)  =0      (V   denotes  the  gradient  vector 

with  respect  to  x) 

A2.    g. (x*)  £0  i  =  1,. ..,m 

m 


A3.     I      X*g..(x*)  =  0 
i=l 


T    0  r"» 

A4.    z  V  L(x*,X*)z  <  0   for  each  nonzero   z  e  E   satisfying 

X  T 

z  V  g.(x*)  =  0  for  all  i  such  that 
xx 

X*  >  0 

x 

A5.    X*  >  0   for  every   i  such  that  g.(x*)  =  0. 

Then  there  exists  a  K  >  0  such  that  for  every  k*  >  K,   x*   is  an 
unconstrained  isolated  local  maximizer  of  P(x,k*,X*). 


Note  that  conditions  Al ,   A2,   A3  of  the  theorem  are  just 
the  Kuhn  Tucker  necessary  conditions,  while,  in  the  presence  of  the 
nondegeneracy  condition  A5,   the  conditions  Al,   A2,   A3,   A4  are 
equivalent  to  the  second  order  sufficiency  conditions  given  by  Fiacco 
and  McCormick  [3]. 

Theorem  1  implies  that  if  the  optimal  Lagrange  multipliers 
X*  are  known,  and  if  k*   is  chosen  large  enough,  then  the  function 
P(x,k*,X*)   can  be  considered  to  be  an  exact  penalty  function.   It 


is  easy  to  see  that  P  has  as  many  orders  of  continuous  derivatives 
as  the  functions   f   and  g.   have.   Of  course,  in  solving  a  nonlinear 
program,  we  do  not  know  the  optimal  multipliers  in  advance. 

In  this  paper  we  will  address  the  question  of  how  to  find 
the  optimal  multipliers  X*     using  an  iterative  scheme.   Through- 
out we  will  assume  that  k*   is  fixed  sufficiently  large  (note  that 
in  the  computations  reported  in  section  V,  moderate  values  of  k* 
were  always  sufficient) . 

b .   Recognizing  an  optimal  solution. 

Suppose  that  some  value  of   X  has  been  chosen.   If  this  X      is 
the  optimal  Lagrange  multiplier  vector   A*,   then  given  the  conditions 
of  theorem  1,  there  exists  a  local  maximum  x*   of  P(x,k,X)   which 
solves  NLP  (1) .   The  following  lemmas  will  enable  us  to  recognize 
when  this  has  occurred. 

Lemma  2 .   Suppose  x*   is  a  local  max  for  P(x,k,X)   (for  some  fixed 
values  of   X  ^  0,   k  >  0)   then  the  Kuhn  Tucker  conditions  are  satis- 
fied at   (x*,A)   if  and  only  if  x*   is  a  local  optimal  solution  to 
NLP. 

Proof.   Necessity  of  the  Kuhn  Tucker  conditions  for  a  solution  to 
NLP  has  been  assumed  from  the  beginning.   To  prove  that  the  conditions 
are  sufficient  we  argue  as  follows.   Since  x*   is  a  local  maximum 
for  P(x,k,X),   3  6  >  0   such  that  ¥  x  e  N  (x*)   we  know 

P(x*,k,X)  :>  P(x,k,X)  (3) 


Let  x  be  any  point  in  N~(x*)  0  S.   We  want  to  show  f(x*)  ^  f(x) 
From  (3) 

P(x*,k,8)  £  P(x,k,B) 
Then  using  the  definition  of  P   in  (2) , 


m 
f(x*)  -  (1/k)  I     X  [exp(k  g  (x*))  -  1]  :> 

i=l 


A  m 

f(x)  -  (1/k)  I     X  [exp(k  g  (x))  -  1]     (4) 
i=l 


Now 

m 

I      X.[exp  k  g  (x*)  -  1]  =  0  (5) 

i=l   1 


since  by  complementary  slackness  if  X.  >  0   then  g.(x*)  ■  0. 


Also 


m 

I     X.[exp  k  g  (x)  -  1]  *  0  (6) 

i=l 


since  X .  ^  0  and  feasibility  of  x  implies   exp(k  g.(x))  -1^0  Vi. 
Combining  (4) ,  (5) ,  and  (6)  we  have 

f(x*)  :>  f(x). 

Thus  since  x*   is  feasible,   x*   is  a  local  optimal  solution  for 
NLP  (1). 

□ 


In  the  above  sufficiency  proof,  only  part  of  the  Kuhn  Tucker 
conditions  were  required.   In  particular  we  did  not  need  to  use  the 
fact  that  V  L(x*,A)  =  0.   Thus  a  more  economical  way  of  writing 
Lemma  2  is  as  follows : 

Lemma  3.   For  fixed   A  e  E  ,   k  >  0,   if 

i)   x*   is  a  local  unconstrained  max  for  P(x,k,A) 
ii)   Aigi(x*)  =0     i  =  l,...,m 
iii)   g.(x*)  £0       i  =  l,...,m 

Then  x*   is  a  local  maximum  for  the  nonlinear  program  NLP  (1) . 

Proof.   Same  as  for  Lemma  2. 

This  result  is  interesting  for  two  reasons.   First,  it  provides 
a  condition  (x*  maximizes  P(x,k,A))   under  which  the  Kuhn  Tucker 
Conditions  are  necessary  and  sufficient  for  x*   to  be  a  local  max 
of  NLP  (1) .   This  is  true  for  any  nonlinear  program  satisfying  a 
constraint  qualification — no  convexity  properties  are  required. 

Second,  Lemma  3  provides  a  termination  condition  for  an  itera- 
tive scheme  of  determining  the  optimal  multipliers   A*. 

c)   Iterative  determination  of  optimal  multipliers. 

Suppose  A   is  not  an  optimal  multiplier  vector.   Then  we  face 
the  problem  of  choosing  better  values  for  the  multipliers.   In  this 
paper  we  will  consider  a  single  iterative  scheme  for  improving  the 
values  of  A.  The  procedure  is  as  follows: 


-1       _ 

1.  Choose  X   e  E  .   Set   s  =  1  as  an  iteration  counter. 

Fix  k  >  0. 

s    n  s 

2.  Let  x  e  E   be  a  local  maximum  for  P(x,k,X  )   obtained 

s-1 
by  starting  from  x     and  using  an  unconstrained 

optimizer.   If  the  conditions   ii,   iii  of  Lemma  3  are 

s 
satisfied,  STOP:   x   is  optimal. 

3.  Let   X    e  E   have  i    component 

Ai+1  =  xi  exp(k  §i(xS))  (7) 

Replace  s  by  s  +  1  and  go  to  step  2. 

An  intuitive  justification  for  the  correction  in  (7)  can  be 

g 
given  for  an  NLP  with  one  constraint.   In  this  case,  if  X   is 

larger  than  the  optimal  X*,   then  we  expect  the  maximizing  x  to 

strictly  satisfy  the  constraint,   g(x)  <  0.   Hence 

, S+l     S      .    ,    v    . s 
X    =  X   exp  k  g(x)  <  X 

g 
so   X   is  made  smaller  for  the  next  iteration.   Similarly  if  X   is 

smaller  than  the  optimal   X*   then  we  expect  the  maximizing  x  to 

violate  the  constraint.   Thus  in  this  case 

, S+l      S       .    t    v     , s 

X    =  X   exp  k  g(x)  >  X 

and  again  X   is  corrected  in  the  appropriate  direction.   Of  course, 
when  there  are  several  constraints,  the  interaction  between  them  makes 
the  above  intuitive  reasoning  less  clear.   Mathematical  justification 
for  the  correction  (7)  will  be  given  later. 


Since  A   at  each  iteration  is  computed  from  (7)  and  since 

s+1 
exp  k  g.(x)  >  0   for  any  x,   the  resulting  A.    will  never  equal 

zero  unless   A.  =  0. 

l 

Practically,  however,  if   g . (x)  <  0,   exp  k  g . (x)   may  be  very 

s+1 
small,  so  that  after  several  cycles  X.  will  be  indistinguishable 

from  zero  on  a  computer  with  finite  word  length.   Note  also  that 

X.    ^  0   is  guaranteed  automatically  by  the  formula  (7)  at  all 

iterations. 

s  s  s 

Suppose  X        has  been  given,  and  that  x   maximizes  P(x,k,A  ). 

s 
If  x   solves  NLP  (1)   then  the  Kuhn  Tucker  conditions  will  be 

3 

satisfied  and  the  algorithm  will  stop.   If,  however,   x   is  not 
optimal,  then  by  Lemma  2  the  Kuhn  Tucker  conditions  cannot  be  satisfied. 
In  this  case  the  method  computes  new  multipliers  using  (7) 

Xi+1  =  Ai  exp(k  §i(xS))  (8) 

Note  that  if  A.  $   0   and  if   g.(x  )  j-   0   then  the  new  multiplier 

s+1  s 

X.  will  be  different  from  A..   The  only  conditions  in  which 

s+1    s 
X.        =A.,   1=1, ...,m  can  occur  are 
l      i 

1.   xS   is  feasible  and  A.g.(x  )  =  0  ¥i  (but  then  by  Lemma  3 
x   is  optimal) 

or     2.   for  some  i  g . (x  )  >  0  while  X.    =  0.   In  this  case, 

s  th         . 

when  A .  =  0   the  penalty  term  for  the  i    constraint 

drops  out  of  the  P   function.   We  can  avoid  this  by  resetting 

A.     to  some  positive  quantity  if  g . (x  )  >  0   and  A.  =  0. 


10 


(In  our  computations  to  date  it  has  never  been  necessary 
to  do  this,  but  for  finite  word  length  computations  it  is 
a  possibility.) 

If  the  above  precaution  is  taken,  then  the  algorithm  will  never  fail 
to  generate  a  different  X  vector  at  each  iteration  unless  an 
optimal  solution  has  been  reached. 


11 


III.  Dual  Feasibility. 

The  formula  (7)  chosen  to  correct  the  Lagrange  multipliers 
is  motivated  by  the  following  duality  considerations.   Note  that 

V  L(xS,AS+1)    =   V   f(xS)    -   I   XS+1   V   g.(xS) 

X  X  u        1  XI 

=  V   f(xS)    -  I   \S   exp(k  g.(xS))V   g    (xS) 

A  1  1X1 

=  Vx  P(xS,k,XS)    =  0  (9) 

s  s 

since  P(x,k,X  )   has  an  unconstrained  max  at  x  .   Hence  at  each 

s   s+1 
stage  of  the  algorithm,  the  point   (x  ,X    )   gives  a  stationary  point 

for  the  Lagrangian  function.   The  method  can  thus  be  viewed  as  one 

s+1 
which  chooses   X     to  minimize  the  error  in  the  Kuhn  Tucker 

condition  V  L(x,X)  =  0. 
x 

If  NLP  (1)   is  a  concave  program,  that  is  if   f   is  concave 

and  each  g.   is  convex,  then  the  Lagrangian  function  is  concave  in 

s+1 
x  for  fixed  X.   In  this  case  (9)  above  and  X    ^0  show  that 

s   s+1 
the  point   (x  ,X    )   is  feasible  for  the  Wolfe  dual  problem  to  NLP 


(Dual)         maximize       L(x,X) 

subject  to     V  L(x,X)  =  0  (10) 

X  £  0 

,  s  , s+lx 
Since  the  Lagrangian  is  concave  in  x,   the  stationary  point  at   (x  ,X   ) 

s+1 
must  be  a  maximum  of  L(x,X   )   with  respect  to  x.   We  will  use  this 

fact  in  proving  convergence  of  the  method  for  concave  programs. 


12 


IV.   Convergence. 

In  section  III  it  was  shown  that  the  formula  (7)  will  always 
generate  a  new  vector  of  multipliers   X  unless  the  optimum  solution 
has  been  reached.   In  this  section  we  show  that  this  sequence  of 
vectors  (and  hence  the  sequence  of  x  vectors  generated  by  them)  is 
an  improving  sequence,  so  that  the  method  will  converge.   We  have 
been  able  to  demonstrate  convergence  only  for  concave  programs,  but 
in  the  limited  computational  tests  performed  to  date,  the  method  has 
always  converged  for  non-concave  programs  also. 

It  should  be  noted  that  for  concave  programs,  the  ordinary 
Lagrangian  function  is  an  exact  penalty  function  if  the  optimal 
multipliers  are  known,  and  iterative  methods  for  obtaining  the  multi- 
pliers are  also  available.   For  non-concave  programs,  however,  methods 
using  the  Lagrangian  as  a  penalty  function  are  known  not  to  converge. 
(In  particular  they  will  fail  on  some  of  the  simple  examples  presented 
in  section  V.) 

The  advantage  of  the  P  function  for  nonconcave  programs  may 
be  explained  by  its  saddlepoint  properties.  A  function  F(x,X)  is 
said  to  have  a  saddlepoint  at   (x*,X*)   if 

F(x,X*)  £  F(x*,X*)  £  F(x*,A)  (11) 

for  all  x  e  E  ,   X  e  E  .   For  concave  programs  both  the  Lagrangian 
L(x,X)   and  the  exact  penalty  function  P(x,k,X)   (with  k  held  fixed) 
are  concave  in  x  for  fixed  X  ^  0  and  linear  in  X   for  fixed  x. 
Hence,  a  saddlepoint  exists  at  the  optimal  solution  to  the  NLP  (1). 


13 


If  the  NLP  (1)  is  not  a  concave  program,  then  the  Lagrangian 
function  will  generally  not  have  a  saddlepoint  at  the  solution   (x*,X*) 
to  the  NLP.   However,  as  shown  by  Gould  and  Howe  [5],  the  P   function 
will  always  have  a  local  saddlepoint  at   (x*,X*)   (for  k  chosen 
sufficiently  large  and  under  the  conditions  of  Theorem  1) . 

To  establish  convergence,  we  first  prove  the  following: 

s   s 
Lemma  4 .   Let  x  ,  X   be  generated  by  the  method  of  section  lie.  for 

s  =  1,2,...,   and  suppose  the  complementary  slackness  condition 

s     s 
X.g.(x)   =  0  (i  =  l,...,m)   is  not  satisfied.   Then 

L(xS,XS+1)  <  L(xS,XS)     s  =  1,2,...  . 


m 
Proof.   L(xS,XS+1)  =  f(xS)  -  I     xf"1  g.(xS) 

i=l 


=  f(xS)  -  I      A   exp(k  g  (x  ))g  (x  )        (12) 
i=l 


s 


from  the  definition  of  the  Lagrangian  and  from  (7).   Now  X.  ^  0  for 


i 


s, 


all   i  =  l,...,m.   If  g.(x  )  >  0   then 


x 

Svv    /  Sx       /  s, 


exp(k  gi(x0))gi(x0)  >  g.(x°)  >  0  (13) 


while  if   g.(x  )  <  0,   then 


g.(xS)  <  exp(k  g±(x  ))gL(x  )  <  0, 


(14) 


Finally,  if  g.(xS)  =  0   then 


14 


g.(xS)  =  exp(k  gi(xS))gi(xS)  =  0  (15) 

In  each  case  (13) ,  (14)  and  (15) , 

X^gi(xS)  £  \S±   exp(k  g.(xS))gi(xS)  (16) 

and  since  complementary  slackness  does  not  hold,  the  inequality  in 

(16)  is  strict  for  at  least  one  i. 

Hence, 

m  m 

f(xS)  -  I  XS±   exp(k  gi(xS))gi(xS)  <  f(xS)  -  I      X^g.(xS) 

i=l  i=l 


that  is,  L(xS,XS+1)  <  L(xS,XS)  (17) 


which  completes  the  proof . 


Lemma  5.   If  NLP  (1)   is  a  concave  program,  then  for  all   s, 


□ 


_,  s+1  ,  s+1.    T/  s  ,s+L  /10X 

L(x   ,X   )  £.   L(x  ,X   )  (18) 


s   s+1  s+1 

Proof.   V  L(x  ,X   )  =  0  by  definition  of   X     as  shown  in  section 

III.   If  NLP  (1)   is  a  concave  program,  then  L(x,X)   is  a  concave 

s  s+1 
function  of  x  for  fixed  X  ^  0.   Thus  the  stationary  point   (x  ,X   ) 

must  be  a  maximum  of  L  with  respect  to  x.   Hence 

_  ,   ,  s+1.    T  ,  s  ,  s+1. 
L(x,X    )  £  L(x  ,X    ) 

for  all  x  e  E  ,   and  letting  x  =  x     completes  the  proof. 


□ 


15 


As  a  consequence  of  lemmas  4  and  5  we  know  that  for  a  concave 
program 

LCx1,*1)  >  Ux1,*2)  £  L(x2,X2)    ...  ^  L(xS,XS)  >  L(xS,AS+1)  ^  ...    (19) 

s   s+1 
as  long  as  a  solution  has  not  been  reached.   Since  the  points   (x  ,  A    ) 

are  feasible  for  the  dual  (10)  and  since  the  dual  objective  is  always 

greater  than  the  primal  objective,  we  know  that  the  entire  decreasing 

sequence  (19)  is  bounded  below  by  the  optimal  value  f(x*)   for  NLP  (1) . 

For  a  concave  problem  the  method  can  thus  be  seen  to  be  a  descent 

method  applied  to  the  dual  problem. 

It  is  interesting  to  observe  that  the  penalty  function  P  which 

we  are  actually  optimizing  in  the  method  increases  when  x  values 

are  changed.   It  is  also  true  that  VxeE,   V  A  e  E  ,   L(x,A)  ^ 

P(x,k,A)   so  that  the  following  diagram  can  be  drawn  to  describe  the 

progress  of  the  method  for  concave  programs. 


f(x*) 
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Convergence  of  the  method  can  be  proved  for  concave  programs 
using  a  standard  convergence  theorem  from  Zangwill  [10].   The  algo- 
rithmic map  is  given  by 

A(xS,XS)  =  (xS,XS+1)  (20) 

.,   s  ,  s+1.    .  s+1  ,  s+1.  /olN 

A(x  ,X   )  =  (x    ,X   )  (21) 

and  the  Lagrangian  function  L(x,X)   serves  as  the  adaption  function. 

s+1 
For   X    >  0,   the  map  A  is  continuous  and  hence  a  closed  map. 

Then  (19)  shows  that  the  adaption  function  improves  at  every  iteration 

(with  strict  improvement  every  second  step)  so  that  convergence 

follows  from  Zangwill' s  results.   This  can  be  summarized  by 

Theorem  6 .   If  NLP  (1)   is  a  concave  program,  then  the  method  of 
section  lie.  will  converge  to  the  optimal  solution   (x*,X*). 

Proof.   (as  above) . 
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V.   Computational  Experience. 

An  experimental  computer  program  has  been  written  to  gain 
experience  with  the  basic  method  of  section  II.   The  program  is 
written  in  FORTRAN  IV  and  the  experiemental  runs  were  done  on  the 
IBM  360  at  the  Naval  Postgraduate  School.   In  its  current  state  the 
program  is  intended  to  test  the  efficacy  of  our  strategy  for  itera- 
tively  determining  the  optimal  multiplier  vector.   No  attempt  has 
been  made  to  optimize  the  code  either  in  terms  of  running  time  or 
function  evaluations  although  there  are  many  opportunities  for  doing 
so  at  the  expense  of  more  complicated  coding. 

In  the  process  of  debugging  the  program  several  small  problems 
were  solved.   Efforts  are  now  continuing  with  larger  and  more  complex 
problems. 

Problem  1.  max  -x  -  x 

2 
subject  to   x.  -  x_  £  0 
J  12 

-x       £  0 

This  is  a  concave  program  given  as  an  example  by  Fiacco  and  McCormick 
[3].   The  optimal  solution  is  at  x  =  x  =  0. 

3 
Problem  2.  max   x 

subject  to   x  -  2  £  0 

Problem  2  is  not  a  concave  program,  but  it  has  a  single  local  max  at 
x  -   2.   The  KTC  are  also  satisfied  at  x  =  0.   The  Lagrangian  function 
does  not  have  a  saddlepoint  at  x  =  2. 
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Problem  3.  max   xixo 

2 
subject  to   xi  +  x_  -  1  £  0 

"xl  "  x2     *  ° 

Problem  3  is  not  concave,  but  it  has  a  single  local  max  at  x.  =  2/3, 


x2  =  /3/3. 


2 
Problem  4.  max   2  x  +  x 

subject  to   2  x.  +  x_  £  2 
J  12 

x  £  0,   x-  ^  0 


Problem  4  has  a  local  max  at  x.  =  1,  x~  =  0  and  the  global  max 
at  x1  =  0,  x  =  2.  As  will  be  seen,  the  method  may  converge  to 
either  of  these  depending  on  the  starting  point. 


Problem  5.  max   x  x~x 

subject   to        x,    +  2   x0  +  2  x„  £  72 
J  12  3 

0  £  x-  £  42 
0  £  x  <.  42 
0  s£  x,  £  42 

Rosenb rock's  post  office  parcel  problem  has  optimal  solution  x..  =24, 
x_  -  12,  x  =  12. 
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Problem  6.  min   (x^l) (x  -2) (x  -3)  +  x 

2    2    2 
subject  to   x  +  x  -  x  £  0 

/     2    2    2    n 
4  -  x   -  x  -  x  ^  0 

x  -  5  £  0 

x  ^  0,   x  ^  0,   x  ;>  0 

Problem  6,  formulated  at  Research  Analysis  Corporation,  has  a  nonconvex 
objective  function  and  nonconvex  feasible  region.  The  optimal  solution 
occurs  at  x  =0,   x  =  /I,   x  -  Jl. 


Problem  7. 


5  5        5 


mm 


y  e.x.  +  y    y  x.c. .x.  +  y  d.x: 

3=1     3    J        i=l  j=l      X  XJ    J        j=l     J    J 


subject   to  y      a..x.^b.  1=1,..., 

j=l        1J    J  1 


10 


x.   ;>  0  j   =  1,. . .  ,5 

J 

Problem  7,  with  coefficients  given  in  Mylander  et  al  [7]  is  due  to 
the  Shell  Development  Company.   It  is  a  convex  program.   Problem  7 
and  its  dual  Problem  8  were  used  as  test  problems  in  Colville's 
comparison  of  NLP  codes  [1] . 
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Problem  8. 

10  5        5  5 

max        y     b.y.    -     T        7     x.c.x.  -  2      T     d.x. 

j»l      J    J         i=l   j=l  J    J  i=l 

10  5  2 

subject   to        £     a^-jy.;   ^  e,-   +  2     £     c4,-x^   +  3  d.x.  i  =  1,...,5 


:t   to        T     a..y.   £  e.   +  2     Y     c..x.   + 

ii       J1  J  i  ii       J1   3 

J=l      J  j=l      J      J 


x.   ^  0  i  «  1.....5 

y     ^  0  j   =  1,...,10 

Problem  8  is  the  dual  of  problem  7.   It  is  not  a  concave  program, 
and  the  feasible  region  is  not  a  convex  set. 

Results  of  the  algorithm  as  applied  to  these  test  problems  are 

given  in  table  1.   As  a  termination  condition  we  required  the  Kuhn 

—ft 
Tucker  Conditions  to  be  satisfied  to  within  10   ,   that  is, 

m  9     -ft 

I     max(g.(x),0)  a:  10 
i=l      1 

m  ?       -ft 

I      (X,g,(x)r    *  10 
i=l 

I    i^^1)2       sio"6. 

J=l       J 

-4 
In  all  cases  this  led  to  solution  values  accurate  to  within  10 

of  the  theoretical  optimum.   For  convenience  all  X  .      were  initially 

set  to  10.0  for  all  problems. 

Note  that  all  problems  were  solved  successfully  for  moderate 

values  of  k   (5  or  10) .   In  general  the  larger  values  of  k  gave 
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Problem 

1 


TABLE  1 
Results  of  Computational  Tests 


Vari-   Con- 
ables  straints 


15 


15 


20 


3   Initial  x 

k 

No. 

iterations 

(1,1) 

5 

7 

(1,1) 

10 

6 

(0) 

5 

7 

(0) 

10 

5 

(1,1) 

1 

Did 

not  converge 

(1,1) 

5 

5 

(1,1) 

10 

4 

(0,0) 

5 

17 

(global) 

(0,0) 

10 

7 

(global) 

(0.5,0) 

10 

7 

(local) 

(10,10,10) 

5 

4 

(10,10,10) 

10 

4 

(0,0,3) 

5 

7 

(0,0,3) 

10 

5 

(.1,.1,.2,.5, 

.5) 

5 

13 

(,.±,.±,.Z,.J, 

.5) 

10 

10 

x.  =  10  4(j^7) 
x  =  60.0 


33 
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faster  convergence  in  terms  of  the  number  of  unconstrained  optimiza- 
tions required,  but  it  should  be  noted  that  if  k  increases,  then 
the  function  P(x,k,X)   will  be  harder  to  optimize  due  to  the 
presence  of  sharp  corners  in  the  exp(k  g.(x))   terms.   In  problem 
4  the  method  converged  either  to  a  local  max  or  to  the  global  max 
depending  on  the  initial  x  values. 
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VI.   Conclusions  and  Extensions. 

The  computational  results  reported  in  section  V  indicate  that 
our  method  has  some  promise  as  a  tool  for  solving  nonlinear  programs. 
There  are  still,  however,  many  improvements  which  must  be  made  before 
routine  use  is  possible. 

The  first  problem  with  the  basic  method  is  that  for  each  itera- 
tion reported  in  section  V  a  complete  unconstrained  optimization  is 
required.   It  would  be  better  if  the  method  could  be  modified  so 
that  revised  A   estimates  are  made  more  frequently.   We  are  currently 
investigating  this  problem. 

The  question  of  convergence  for  non-concave  problems  is  not 
yet  resolved.   Further  computational  tests  and  theoretical  work  are 
planned  to  help  answer  this  question. 

The  major  goal  which  has  been  achieved  is  the  development  of 
a  sequential  unconstrained  penalty  function  technique  in  which  the 
successive  unconstrained  problems  do  not  become  increasingly  ill 
conditioned.   Hopefully,  this  will  tend  to  alleviate  the  numerical 
problems  encountered  in  other  penalty  function  methods. 
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